Neural Networks with À La Carte Selection of Activation Functions
نویسندگان
چکیده
Activation functions (AFs), which are pivotal to the success (or failure) of a neural network, have received increased attention in recent years, with researchers seeking design novel AFs that improve some aspect network performance. In this paper we take another direction, wherein combine slew known into successful architectures, proposing three methods do so beneficially: (1) generate AF architectures at random, (2) use Optuna, an automatic hyper-parameter optimization software framework, Tree-structured Parzen Estimator (TPE) sampler, and (3) Optuna Covariance Matrix Adaptation Evolution Strategy (CMA-ES) sampler. We show all often produce significantly better results for 25 classification problems when compared standard composed ReLU hidden units softmax output unit. TPE sampler emerged as best architecture-producing method.
منابع مشابه
Death à la carte
Delegates arrived at the hotel on a rainy Sunday evening where they were immediately made welcome and encouraged to relax over dinner while chatting to other EWCD (European Workshop on Cell Death) participants. The EWCD is a bi-annual conference that focuses on all aspects of cell death. Each conference is held in a different unique location, with this year’s being no exception. The six-day mee...
متن کاملConcurrent Objects à la Carte
Services are autonomous, self-describing, technology-neutral software units that can be described, published, discovered, and composed into software applications at run-time. Designing software services and composing services in order to form applications or composite services requires abstractions beyond those found in typical object-oriented programming languages. In this paper, we explore a ...
متن کاملFishing history à la carte
The decline in many marine fisheries is well documented but there is less information about the scale and timing of their demise. Researchers are now beginning to search for such information in the hope that it may help reveal how different fisheries might best be conserved or restored. One novel source has been seafood prices collected from US restaurant menus dating to the 1850s, which are he...
متن کاملPolarization response of nanowires à la carte
Thanks to their special interaction with light, semiconductor nanowires have opened new avenues in photonics, quantum optics and solar energy harvesting. One of the major challenges for their full technological deployment has been their strong polarization dependence in light absorption and emission. In the past, metal nanostructures have been shown to have the ability to modify and enhance the...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: SN computer science
سال: 2021
ISSN: ['2661-8907', '2662-995X']
DOI: https://doi.org/10.1007/s42979-021-00885-1